Seam carving for image resizing

ABSTRACT

A method for modifying an input digital image having input dimensions defined by a number of input rows and input columns to form an output digital image where the number of rows or columns is reduced by one, comprising an image energy map determined from the input image; determining a seam path responsive to the image energy map; imposing constraints on the seam path; and removing pixels along the seam path to modify the input digital image.

FIELD OF THE INVENTION

This invention relates generally to the field of resizing digitalimages, and more particularly to a method to resize images using a seamcarving algorithm.

BACKGROUND OF THE INVENTION

The rise of print and display devices ranging from tiny “thumbprints” ofimages often seen in selection menus, small, low resolution mobiletelephone screens, slightly larger PDA screens, to large, highresolution elongated flat panel display and projector screens has madeimage resizing an important technique for rendering and viewing digitalimages. Resizing images to render them on different devices thanoriginally intended is sometimes called image retargeting.

Conventional image retargeting typically involves image scaling andcropping. Image scaling magnifies or shrinks the size of the image toresize the image. Generally, the same scale factor is applied in boththe horizontal and vertical directions, which preserves the aspect ratioof the image. Image scaling alone does not work well when the aspectratio of the image needs to change, because applying different scalefactors in the horizontal and vertical directions introduces visualdistortions.

Cropping is another method to resize an image by cutting out a subset ofpixels within the image. Generally, image scaling is combined withcropping when the aspect ratio of an image needs to be changed. In thiscase, the image is scaled so that it has the right size in onedimension, but is oversized in the other direction. The scaled image isthen cropped to obtain an output image of the desired size.

Many resizing algorithms default to cropping the output image from thecentral portion of the input image, discarding equal portions of theinput image on both edges. However, this can result in discardingimportant parts of the image depending on the content of the image.While cropping an image, there is a desire to maintain importantfeatures in the content of the image. This can be done with top-down orbottom-up methods. Top-down methods use tools such as face detectors todetect important regions in the image, whereas bottom-up methods rely onvisual saliency methods to construct visual saliency map of the sourceimage. After the saliency map is constructed, cropping can be used todisplay the most important region of the image.

One method described by Suh et al., in the article “Automatic thumbnailcropping and its effectiveness” (Proceedings of the 16th annual ACMsymposium on User Interface Software and Technology, pp. 95-104, 2003)automatically generates thumbnail images based on either a saliency mapor the output of a face detector. With this method, a source image iscropped to capture the most salient region in the image.

Another method taught by Chen et al. in the article “A visual attentionmodel for adapting images on small displays” (Multimedia Systems, Vol.9, pp. 353-364, 2003) adapts images to mobile devices. In this method,the most important region in the image is automatically detected andtransmitted to the mobile device.

Santella et al., in the article “Gaze-based interaction forsemiautomatic photo cropping” (ACM Human Factors in Computing Systems,pp. 771-780, 2006), which is incorporated herein by reference, use eyetracking, in addition to composition rules to crop images intelligently.In this method, a users looks at an image, while eye movements arerecorded. The recordings are used to identify important image content,and can then automatically crop the image to any size or aspect ratio.

All of the above rely on conventional image resizing and croppingoperations to retarget of the image. These approaches are limitedbecause it can only remove pixels from the image periphery. In somecases, there may be important image content at the edges of the imagethat will be lost during the cropping operation no matter how the imageis cropped. More effective resizing can only be achieved by consideringthe image content as a whole, in conjunction with geometric constraintsof the output device.

Another method taught by Gal et al. in the article “Feature awaretexturing” (Proc. Eurographics Symposium on Rendering, 2006) uses afeature-aware texture mapping that warps an image to a new shape, whilepreserving user-specified regions. This is accomplished by solving aparticular formulation of the Laplace editing technique suited toaccommodate similarity constraints in images. However, local constraintsare propagated through the entire image to accommodate all constraintsat once, and may sometimes fail.

Another method taught by Agarwala et al. in the article “Interactivedigital photomontage” (ACM Trans. Graph. Vol. 23, pp. 294-302, 2004)composes a novel photomontage from several images. A user selects ROIsfrom different input images, which are then composited into an outputimage.

One rather elegant content-aware image retargeting algorithm called“seam carving” has been described by S. Avidan and A. Shamir in U.S.Patent Application Publication 2008/0219587, entitled “Method forretargeting images.” The seam carving technique provides a way tosystematically remove pixels from visually “unimportant” paths (“seams”)through an image, effectively reducing the height or width by one pixelat a time, in a relatively unnoticeable way. Similarly, pixels can beadded to these paths to achieve an increase in the dimension. However,this approach fails if seam passes through the important objects in theimage.

Another method using mesh parameterization has been described by Y. Guoet al. in the article “Image retargeting using mesh parameterization,”(IEEE Transactions on Multimedia, Vol. 11, pp. 856-867, 2009). In thisapproach, a mesh image representation that is consistent with theunderlying image structures is constructed for image retargeting. Thistechnique requires processing an entire image at once which may be toocomplex and too costly for many applications.

Another method described by D. Simakov, et al. in the article“Summarizing visual data using bidirectional similarity” (Proc. IEEEConference on Computer Vision and Pattern Recognition, pp. 1-8, 2008)uses a similarity measure. In this approach, an image similarity measureis optimized for image retargeting. This technique requires processingan entire image at once which may be too complex and too costly for manyapplications.

Thus, there exists a need for content-aware image retargeting thatpreserves salient features of an image even under arbitrary changing ofthe aspect ratio.

SUMMARY OF THE INVENTION

The present invention represents a method for modifying an input digitalimage having input dimensions defined by a number of input rows andinput columns to form an output digital image where the number of rowsor columns is reduced by one, comprising using a processor to performthe steps of

a) selecting either a horizontal seam direction to remove a row or avertical seam direction to remove a column;

b) determining an image energy map for the input image;

c) defining a gradient threshold;

d) determining a seam path responsive to the image energy map, where theseam path is constrained such that a directional image gradient computedin a direction perpendicular to either the seam path or the seamdirection is constrained to be less than the gradient threshold for eachpixel in the seam path; and

e) determining an output digital image by removing pixels along the seampath.

It is an advantage that by constraining the seam path, the retargetedimage preserves the salient features of the original image with improvedaccuracy.

In addition to the embodiments described above, further embodiments willbecome apparent by reference to the drawings and by study of thefollowing detailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be more readily understood from the detaileddescription of exemplary embodiments presented below considered inconjunction with the attached drawings, of which:

FIG. 1 is a high-level diagram showing the components of a system forretargeting digital image according to an embodiment of the presentinvention;

FIG. 2 is a flow diagram illustrating a method for retargeting digitalimages according to an embodiment of the present invention;

FIG. 3 is a block diagram showing a detailed view of the constrain seampath step of FIG. 2;

FIG. 4 is a block diagram showing a detailed view of the constrain seampath step of FIG. 2 for an alternate embodiment of the presentinvention; and

FIG. 5 shows images comparing the results obtained according to anembodiment of the present invention with those obtained using analternate technique.

DETAILED DESCRIPTION OF THE INVENTION

The invention is inclusive of combinations of the embodiments describedherein. References to “a particular embodiment” and the like refer tofeatures that are present in at least one embodiment of the invention.Separate references to “an embodiment” or “particular embodiments” orthe like do not necessarily refer to the same embodiment or embodiments;however, such embodiments are not mutually exclusive, unless soindicated or as are readily apparent to one of skill in the art. The useof singular or plural in referring to the “method” or “methods” and thelike is not limiting.

The phrase, “digital content record”, as used herein, refers to anydigital content record, such as a digital still image, a digital audiofile, or a digital video file.

It should be noted that, unless otherwise explicitly noted or requiredby context, the word “or” is used in this disclosure in a non-exclusivesense.

FIG. 1 is a high-level diagram showing the components of a system forimage retargeting according to an embodiment of the present invention.The system includes a data processing system 110, a peripheral system120, a user interface system 130, and a data storage system 140. Theperipheral system 120, the user interface system 130 and the datastorage system 140 are communicatively connected to the data processingsystem 110.

The data processing system 110 includes one or more data processingdevices that implement the processes of the various embodiments of thepresent invention, including the example processes of FIGS. 2-4described herein. The phrases “data processing device” or “dataprocessor” are intended to include any data processing device, such as acentral processing unit (“CPU”), a desktop computer, a laptop computer,a mainframe computer, a personal digital assistant, a Blackberry™, adigital camera, cellular phone, or any other device for processing data,managing data, or handling data, whether implemented with electrical,magnetic, optical, biological components, or otherwise.

The data storage system 140 includes one or more processor-accessiblememories configured to store information, including the informationneeded to execute the processes of the various embodiments of thepresent invention, including the example processes of FIGS. 2-4described herein. The data storage system 140 may be a distributedprocessor-accessible memory system including multipleprocessor-accessible memories communicatively connected to the dataprocessing system 110 via a plurality of computers or devices. On theother hand, the data storage system 140 need not be a distributedprocessor-accessible memory system and, consequently, may include one ormore processor-accessible memories located within a single dataprocessor or device.

The phrase “processor-accessible memory” is intended to include anyprocessor-accessible data storage device, whether volatile ornonvolatile, electronic, magnetic, optical, or otherwise, including butnot limited to, registers, floppy disks, hard disks, Compact Discs,DVDs, flash memories, ROMs, and RAMs.

The phrase “communicatively connected” is intended to include any typeof connection, whether wired or wireless, between devices, dataprocessors, or programs in which data may be communicated.

The phrase “communicatively connected” is intended to include aconnection between devices or programs within a single data processor, aconnection between devices or programs located in different dataprocessors, and a connection between devices not located in dataprocessors at all. In this regard, although the data storage system 140is shown separately from the data processing system 110, one skilled inthe art will appreciate that the data storage system 140 may be storedcompletely or partially within the data processing system 110. Furtherin this regard, although the peripheral system 120 and the userinterface system 130 are shown separately from the data processingsystem 110, one skilled in the art will appreciate that one or both ofsuch systems may be stored completely or partially within the dataprocessing system 110.

The peripheral system 120 may include one or more devices configured toprovide digital content records to the data processing system 110. Forexample, the peripheral system 120 may include digital still cameras,digital video cameras, cellular phones, or other data processors. Thedata processing system 110, upon receipt of digital content records froma device in the peripheral system 120, may store such digital contentrecords in the data storage system 140.

The user interface system 130 may include a mouse, a keyboard, anothercomputer, or any device or combination of devices from which data isinput to the data processing system 110. In this regard, although theperipheral system 120 is shown separately from the user interface system130, the peripheral system 120 may be included as part of the userinterface system 130.

The user interface system 130 also may include a display device, aprocessor-accessible memory, or any device or combination of devices towhich data is output by the data processing system 110. In this regard,if the user interface system 130 includes a processor-accessible memory,such memory may be part of the data storage system 140 even though theuser interface system 130 and the data storage system 140 are shownseparately in FIG. 1.

FIG. 2 is a flow diagram illustrating a method for retargeting digitalimages according to an embodiment of the present invention. A digitalimage 203 representing a scene is received in receive digital image step202. The digital image 203 can be captured by a digital camera or ascanner. Alternately, it may be a frame of a video sequence captured bya video camera.

Seam direction 205 is received in receive seam direction step 204. Theseam direction 205 contains the information specifying whether a row ora column of the image will be removed. The compute seam path step 206uses the seam direction 205 and the digital image 203 to compute a seampath 207. The constrain seam path step 208 imposes constraints on theseam path 207 to produce a constrained seam path 209. The constrain seampath step 208 is responsive to the digital image 203 and optionally tothe seam direction 205. (Note that optional features shown in this andother figures are represented with dashed lines.) In the final step, theretarget image step 210 removes the pixels along the constrained seampath 209 to produce a retargeted digital image 211.

The individual steps outlined in FIG. 2 will now be described in greaterdetail. The compute seam path step 206 produces a seam path 207. In apreferred embodiment of the present invention, the seam path 207 iscomputed using a seam carving algorithm such as that described in U.S.Patent Application Publication 2008/0219587, by S. Avidan and A. Shamir,entitled “Method for retargeting images,” which is incorporated hereinby reference. With this approach, the seam path 207 represents alowest-energy, connected path of pixels in the direction provided byseam direction 205. The energy of the seam path 207 is computed from animage energy map determined from the digital image 203.

The image energy map can be computed in any appropriate way known tothose skilled in the art. In a preferred embodiment of the presentinvention, the image energy map is computed from the derivatives of thedigital image 203 as described in the aforementioned U.S. PatentApplication Publication No. 2008/0219587, and is given by:

$\begin{matrix}{{{Emap}\left( {x,y} \right)} = {{{\frac{\partial}{\partial x}{I\left( {x,y} \right)}}} + {{\frac{\partial}{\partial y}{I\left( {x,y} \right)}}}}} & (1)\end{matrix}$

where (x,y) is the pixel location, I(x,y) is the pixel value of thedigital image 203 at pixel location (x,y), ∂/∂x and ∂/∂y are the partialderivative operators in the x and y directions, respectively, |●|denotes absolute value operator, and Emap(x,y) is the value of the imageenergy map at pixel location (x,y). For the case where the digital image203 is a color image, the image energy map is preferably determinedusing a luminance image determined from the digital image 203 usingmethods well known in the art.

In a preferred embodiment of the present invention, the energyassociated the seam path of pixels is determined by summing up the imageenergy map values for the connected path of pixels that lie along theseam path.E _(S)=Σ_(S) Emap(x _(S) ,x _(S))  (2)where the summation is carried out for all of the points on a seam path207, (x_(S),y_(S)) is the location of a point on the seam path 207, andE_(S) is the seam path energy. The compute seam path step 206 determinesthe seam path 207 having the minimum seam path energy. This can be doneusing any optimization process known in the art, such as theoptimization process described in the aforementioned U.S. PatentApplication Publication No. 2008/0219587.

Generally, an initial point is selected on the boundary of the image asthe starting location of the seam path 207. The seam path 207 is thenextended from this point in the direction that minimizes the energy,crossing the digital image 203 in the seam direction 205 (FIG. 2). Theinitial point can be a predetermined point such as a corner point of theimage a center point of the one of the edges. Alternately, the initialpoint can also be determined by selecting an edge location having aminimum energy, or corresponding to a row (column) of the image havingthe lowest average energy.

FIG. 3 is a more detailed view of the constrain seam path step 208 shownin FIG. 2 according to a preferred embodiment of the present invention.A compute seam path directional gradient step 302 uses the digital image203 (FIG. 2) and the seam path 207 (FIG. 2) to compute a directionalimage gradient 303 perpendicular to the seam path 207, or alternatelyperpendicular to the seam direction 205. The directional image gradientcan be computed in any appropriate way known to those skilled in theart. One way to compute the directional image gradient according to apreferred embodiment of the present invention can be described using Eq.(3) below:e(x,y)=|I(x,y)−I(x′,y′)|  (3)where (x,y) and (x′,y′) are neighboring pixel locations lying on a lineperpendicular to the seam direction 205 (or the seam path 207), I(x,y)and I(x′,y′) are the pixel values of the digital image 203 (FIG. 2) atpixel locations (x,y) and (x′,y′), respectively, |●| denotes absolutevalue operator, and e(x,y) is the value of the directional imagegradient at pixel location (x,y).

In FIG. 3, the impose seam constraint step 304 uses the directionalimage gradient 303 and the seam path 207 to impose constraints on theseam path 207, producing the constrained seam path 209. In a preferredembodiment of the present invention, the impose seam constraint step 304is implemented by constraining the directional image gradient to be lessthan a gradient threshold 306 for each pixel along the constrained seampath 209. The gradient threshold 306 is a threshold value specifying amaximum allowable value for the directional image gradient 303 along theconstrained seam path 209.

The gradient threshold can be determined in any appropriate way known tothose skilled in the art. In one embodiment of the present invention,the gradient threshold 306 is a predetermined constant. However, it hasbeen found that in many cases, it is preferable for the gradientthreshold 306 to be image dependent. A user interface can be providedallowing the user to specify a heuristically determined gradientthreshold 306 that works best for a particular image. In a preferredembodiment of the present invention, a determine gradient threshold step305 is used to determine the gradient threshold 306 responsive to thedirectional image gradient 303. This can be done by computing acumulative distribution of the directional image gradient 303 and thenselecting a gradient threshold 306 corresponding to a certain cumulativepercentage (e.g., 80%). In this way, the constrained seam path 209 canbe selected to avoid the areas of the digital image 203 that have thehighest directional image gradient 303.

The impose seam constraint step 304 imposes the constraint conditionthat the directional image gradient 303 be less than the gradientthreshold 306 along the constrained seam path 209. First, the seam path207 determined by the compute seam path step 206 is checked to see if itviolates the constraint condition. If the seam path 207 is found tosatisfy the constraint condition, then the constrained seam path 209 isset to be equal to the seam path 207. However, if the seam path 207 isfound to violate the constraint condition, then the seam path 207 isperturbed until a constrained seam path 209 is identified that satisfiesthe constraint condition. In one embodiment of the present invention,the seam path 207 is perturbed by adjusting the starting location forthe seam path 207. If no constrained seam path 209 is identified thatsatisfies the constraint condition, then the gradient threshold 306 canbe increased to ease the constraint condition until an acceptableconstrained seam path 209 can be found.

The retarget image step 210 of FIG. 2 removes the pixels of the digitalimage 203 along the direction of the constrained seam path 209 producedby the constrain seam path step 208 to produce a retargeted digitalimage 211. In one embodiment of the present invention, the pixels alongthe constrained seam path 209 are simply removed and the remaining imagepixels are shifted to close up the seam. However, this can result invisible artifacts along the seam in some cases. In a preferredembodiment of the present invention, the image pixels of the retargeteddigital image 211 in a neighborhood near the image pixels on theconstrained seam path 209 are modified using the values of the removedimage pixels. The pixel values can be modified in any appropriate wayknown to those skilled in the art. In a preferred embodiment of thepresent invention, the pixels of the retargeted digital image 211adjacent to the constrained seam path 209 are replaced by the average ofthe original image pixel values at those locations and the adjacentremoved image pixel values.

FIG. 4 shows an alternate embodiment of the constrain seam path block208 (FIG. 2). A compute image gradient map step 402 uses the digitalimage 203 and the seam direction 205 to compute an image gradient map403 comprising a directional image gradient perpendicular to the seamdirection 205 (or the seam path 207). The image gradient map 403 can becomputed in any appropriate way known to those skilled in the art. In apreferred embodiment of the present invention, the directional imagegradient given in Eq. (3) is used to compute the image gradient map 403.

Next, a determine illegal pixel locations step 404 uses the imagegradient map 403 produced by compute image gradient map step 402 todetermine an illegal pixel locations map 405. The illegal pixellocations can be computed in any appropriate way known to those skilledin the art. One way to compute the illegal pixel locations according toa preferred embodiment of the present invention can be described usingthe following equation:

$\begin{matrix}{{{IM}\left( {x,y} \right)} = \left\{ \begin{matrix}{{0\mspace{14mu}{if}\mspace{14mu}{{\mathbb{e}}\left( {x,y} \right)}} \leq \eta} \\{1\mspace{14mu}{otherwise}}\end{matrix} \right.} & (4)\end{matrix}$where e(x,y) is the image gradient map 403, η is the gradient threshold306, and IM(x,y) is the value of illegal pixel location map whereIM(x,y)=0 indicates legal pixel locations and IM(x,y)=1 indicatesillegal pixel locations. The gradient threshold 306 can be determined inany appropriate way known to those skilled in the art. As describedabove with reference to FIG. 3, in a preferred embodiment of the presentinvention a determine gradient threshold step 305 can be used todetermine the gradient threshold 306 responsive to the image gradientmap 403.

Next, a constrain seam direction step 406 uses the illegal pixellocations map 405 and the seam path 207 to determine the constraintedseam path 209. In a preferred embodiment of the present invention, theconstrain seam path step 406 is implemented by constraining the seampath 207 to not pass through any illegal pixel locations.

The constrain seam path step 406 imposes the constraint that theconstrained seam path 209 not pass through any illegal pixel locations.First, the seam path 207 determined by the compute seam path step 206(FIG. 2) is checked against the illegal pixel locations map 405 to seeif it violates passes through any illegal pixel locations. If the seampath 207 is found to satisfy the constraint condition, then theconstrained seam path 209 is set to be equal to the seam path 207.However, if the seam path 207 is found to pass through any illegal pixellocations, then the seam path 207 is perturbed until a constrained seampath 209 is identified that does not pass through any illegal pixellocations. In one embodiment of the present invention, the seam path 207is perturbed by adjusting the starting location for the seam path 207.If no constrained seam path 209 is identified that satisfies theconstraint condition, then the gradient threshold 306 can be increasedto ease the constraint condition until an acceptable constrained seampath 209 can be found.

The algorithm described above to modifying an input digital image 203 toform a retargeted digital image 211 where the number of rows or columnsis reduced by one can be applied repeatedly to reduce the number of rowsor columns until a desired output aspect ratio is reached. FIG. 5 showsan example of a source digital image 500. A retargeted digital image 510is shown where the number of columns has been reduced by 40% accordingto the present invention. For comparison, a prior art retargeted digitalimage 520 is also shown determined according to the seam carvingalgorithm described in the aforementioned U.S. Patent ApplicationPublication 2008/0219587. It can be seen that the present invention hasthe advantage that important details are persevered in the retargeteddigital image 510 that have been lost in the prior art retargeteddigital image 520.

It is to be understood that the exemplary embodiments disclosed hereinare merely illustrative of the present invention and that manyvariations of the above-described embodiments can be devised by oneskilled in the art without departing from the scope of the invention. Itis therefore intended that all such variations be included within thescope of the following claims and their equivalents.

PARTS LIST

-   110 Data processing system-   120 Peripheral system-   130 User interface system-   140 Data storage system-   202 Receive digital image step-   203 Digital image-   204 Receive seam direction-   205 Seam direction-   206 Compute seam path step-   207 Seam path-   208 Constrain seam path step-   209 Constrained seam path-   210 Retarget image step-   211 Retargeted digital image-   302 Compute seam path directional gradient step-   303 Directional image gradient-   304 Impose seam constraint step-   305 Determine gradient threshold-   306 Gradient threshold-   402 Compute image gradient map step-   403 Image gradient map-   404 Determine illegal pixel locations step-   405 Illegal pixel locations map-   406 Constrain seam path step-   500 Source digital image-   510 Retargeted digital image-   520 Prior art retargeted digital image

1. A method for modifying an input digital image having input dimensionsdefined by a number of input rows and input columns to form an outputdigital image where the number of rows or columns is reduced by one,comprising a) selecting either a horizontal seam direction to remove arow or a vertical seam direction to remove a column; b) determining animage energy map for the input image; c) defining a gradient threshold;d) determining a seam path responsive to the image energy map, where theseam path is constrained such that a directional image gradient computedin a direction perpendicular to either the seam path or the seamdirection is less than the gradient threshold for each pixel in the seampath; and e) determining an output digital image by removing pixelsalong the seam path; wherein the method is performed at least in partusing a data processing system.
 2. The method of claim 1 where step e)includes modifying the output digital image pixels in a neighborhoodnear the seam path using the values of the removed pixels.
 3. The methodof claim 2 where the values of output digital image pixels adjacent tothe seam path are replaced by the average of the original image pixelvalues and the adjacent removed image pixel values.
 4. The method ofclaim 1 wherein the image energy map is computed by:${{Emap}\left( {x,y} \right)} = {{{\frac{\partial}{\partial x}{I\left( {x,y} \right)}}} + {{\frac{\partial}{\partial y}{I\left( {x,y} \right)}}}}$where (x,y) is the pixel location, I(x,y) is the pixel value of thedigital image at pixel location (x,y), ∂/∂x and ∂/∂y are the partialderivative operators in the x and y directions, respectively, |●|denotes absolute value operator, and Emap(x,y) is the value of the imageenergy map at pixel location (x,y).
 5. The method of claim 1 wherein thegradient threshold is determined responsive to a distribution ofdirectional image gradient values.
 6. The method of claim 5 wherein thegradient threshold is determined such that a specified fraction of thedirectional image gradients in the distribution of directional imagegradient values fall below the gradient threshold.
 7. The method ofclaim 1 wherein the directional image gradient is computed in adirection perpendicular to the seam direction and step d) furtherincludes: i) determining an image gradient map where the gradient is adirectional image gradient computed in a direction perpendicular to theseam direction; ii) thresholding the image gradient map with thegradient threshold to determine illegal pixel locations; and iii)determining a seam path responsive to the image energy map, where theseam path does not pass through any illegal pixel locations.
 8. Themethod of claim 1 wherein the input digital image is a color inputdigital image and the seam path is determined using a luminance imagedetermined from the color input image.
 9. A method for resizing an inputdigital image with an input aspect ratio to form an output digital imagewith an output aspect ratio where the method of claim 1 is appliedrepeatedly to reduce the number of input rows or input columns until theoutput aspect ratio is reached.
 10. A system comprising: a dataprocessing system; and a memory system communicatively connected to thedata processing system and storing instructions configured to cause thedata processing system to implement a method for modifying an inputdigital image having input dimensions defined by a number of input rowsand input columns to form an output digital image where the number ofrows or columns is reduced by one, wherein the instructions comprise: a)selecting either a horizontal seam direction to remove a row or avertical seam direction to remove a column; b) determining an imageenergy map for the input image; c) defining a gradient threshold; d)determining a seam path responsive to the image energy map, where theseam path is constrained such that a directional image gradient computedin a direction perpendicular to either the seam path or the seamdirection is less than the gradient threshold for each pixel in the seampath; and e) determining an output digital image by removing pixelsalong the seam path.